Skip to content

feat: optimize user mount cache #57815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Altahrim merged 1 commit into from
Jan 29, 2026
Merged

feat: optimize user mount cache #57815

Altahrim merged 1 commit into from
Jan 29, 2026
+2 −1

Conversation

@Altahrim
Copy link
Collaborator

@Altahrim Altahrim commented Jan 26, 2026 •
edited
Loading

Optimize requests like:

SELECT
  `storage_id`,
  `root_id`,
  `user_id`,
  `mount_point`,
  `mount_id`,
  `f`.`path`,
  `mount_provider_class`
FROM
  `oc_mounts` `m`
  INNER JOIN `oc_filecache` `f` ON `m`.`root_id` = `f`.`fileid`
WHERE
  (`storage_id` = ?)
  AND (
    (`f`.`fileid` = ?)
    OR (`f`.`path` = ?)
    OR (
      `CONCAT` (`f`.`path`, ?) = SUBSTRING (?, ..., `CHAR_LENGTH` (`f`.`path`) + ?)
    )
  )

TODO

  • Is it better to add both fields so DB can choose the best path? ⇒ Changed

Explain

Before

+------+-------------+-------+--------+-----------------------------------------------------------+--------------------+---------+--------------+-------+-------------+
| id   | select_type | table | type   | possible_keys                                             | key                | key_len | ref          | rows  | Extra       |
+------+-------------+-------+--------+-----------------------------------------------------------+--------------------+---------+--------------+-------+-------------+
|    1 | SIMPLE      | m     | ref    | mounts_storage_index,mounts_root_index,mount_user_storage | mount_user_storage | 8       | const        | 64048 |             |
|    1 | SIMPLE      | f     | eq_ref | PRIMARY                                                   | PRIMARY            | 8       | oc.m.root_id | 1     | Using where |
+------+-------------+-------+--------+-----------------------------------------------------------+--------------------+---------+--------------+-------+-------------+

After

+------+-------------+-------+------+-------------------------------------------------------------------------------------------------------------+---------------------+---------+-------------+-------+-------------+
| id   | select_type | table | type | possible_keys                                                                                               | key                 | key_len | ref         | rows  | Extra       |
+------+-------------+-------+------+-------------------------------------------------------------------------------------------------------------+---------------------+---------+-------------+-------+-------------+
|    1 | SIMPLE      | f     | ref  | PRIMARY,fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,fs_storage_path_prefix | fs_storage_mimetype | 8       | const       | 20918 | Using where |
|    1 | SIMPLE      | m     | ref  | mounts_root_index                                                                                           | mounts_root_index   | 8       | oc.f.fileid | 7     |             |
+------+-------------+-------+------+-------------------------------------------------------------------------------------------------------------+---------------------+---------+-------------+-------+-------------+

The change allows to filter on filecache first

(Rough) Benchmark

  • Before: ≈ 400 ms
  • After : ≈ 70 ms

@Altahrim Altahrim added this to the Nextcloud 34 milestone Jan 26, 2026
@Altahrim Altahrim self-assigned this Jan 26, 2026
@Altahrim Altahrim requested a review from a team as a code owner January 26, 2026 12:39
@Altahrim Altahrim added the 3. to review Waiting for reviews label Jan 26, 2026
@Altahrim Altahrim requested review from ArtificialOwl and removed request for a team January 26, 2026 12:39
@icewind1991
Copy link
Member

icewind1991 commented Jan 26, 2026

I would expect the performance comparison to go the other way around for instances that have a few mounts per user but a lot of files. Adding both comparisons and hoping the DB knows what to do is probably for the best.

@Altahrim
Copy link
Collaborator Author

Altahrim commented Jan 26, 2026

On my instance, having both still filter on oc_mounts first and is slower…

nickvergessen
nickvergessen reviewed Jan 26, 2026
View reviewed changes
@nickvergessen nickvergessen requested review from icewind1991 and nickvergessen and removed request for ArtificialOwl and leftybournes January 26, 2026 13:12
@Altahrim Altahrim force-pushed the feat/user-mount-cache-perf branch from e8727f6 to 9560e1c Compare January 26, 2026 13:25
@Altahrim
Copy link
Collaborator Author

Altahrim commented Jan 26, 2026 •
edited
Loading

I was able to trigger the same EXPLAIN by adding a condition to the join: AND m.storage_id = f.storage
Then, I hope the DB will choose wisely :)

@Altahrim
Copy link
Collaborator Author

Altahrim commented Jan 26, 2026 •
edited by nickvergessen
Loading

/backport to stable33

@Altahrim
feat: optimize user mount cache
f71f19c 
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
@Altahrim Altahrim force-pushed the feat/user-mount-cache-perf branch from 9560e1c to f71f19c Compare January 27, 2026 15:31
@Altahrim Altahrim enabled auto-merge January 27, 2026 15:59
@Altahrim Altahrim merged commit 4a077c3 into master Jan 29, 2026
220 of 232 checks passed
@Altahrim Altahrim deleted the feat/user-mount-cache-perf branch January 29, 2026 15:35
@CarlSchwan
Copy link
Member

CarlSchwan commented Jan 29, 2026

/backport to stable33

@backportbot backportbot bot mentioned this pull request Jan 29, 2026
Merged
@backportbot
Copy link

backportbot bot commented Jan 29, 2026

The backport to stable33 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable33
git pull origin stable33

# Create the new backport branch
git checkout -b backport/57815/stable33

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts, resolve them
git cherry-pick f71f19c0

# Push the cherry pick commit to the remote repository and open a pull request
git push origin backport/57815/stable33

Error: Failed to create pull request: Validation Failed: {"resource":"PullRequest","code":"custom","message":"A pull request already exists for nextcloud:backport/57815/stable33."} - https://docs.github.com/rest/pulls/pulls#create-a-pull-request

Learn more about backports at https://docs.nextcloud.com/server/stable/go.php?to=developer-backports.

@Altahrim
Copy link
Collaborator Author

Altahrim commented Jan 29, 2026

Not sure why it was auto-merged because it breaks sharding according to tests…
https://github.com/nextcloud/server/actions/runs/21403298463/job/61620219674

@artonge artonge mentioned this pull request Jan 29, 2026
Merged
@icewind1991 icewind1991 mentioned this pull request Jan 29, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@icewind1991 icewind1991 icewind1991 approved these changes

@CarlSchwan CarlSchwan CarlSchwan approved these changes

@sorbaugh sorbaugh Awaiting requested review from sorbaugh sorbaugh is a code owner automatically assigned from nextcloud/server-backend

@come-nc come-nc Awaiting requested review from come-nc come-nc is a code owner automatically assigned from nextcloud/server-backend

@nickvergessen nickvergessen Awaiting requested review from nickvergessen

Assignees

@Altahrim Altahrim

Labels

3. to review Waiting for reviews feature: filesystem performance 🚀

Projects

None yet

Milestone

Nextcloud 34

Development

Successfully merging this pull request may close these issues.

5 participants

@Altahrim @icewind1991 @CarlSchwan @nickvergessen